Detangling People: Individuating Multiple Close People and Their Body Parts via Region Assembly

摘要

Today's person detection methods work best when people are in common uprightposes and appear reasonably well spaced out in the image. However, in many realimages, that's not what people do. People often appear quite close to eachother, e.g., with limbs linked or heads touching, and their poses are often notpedestrian-like. We propose an approach to detangle people in multi-personimages. We formulate the task as a region assembly problem. Starting from alarge set of overlapping regions from body part semantic segmentation andgeneric object proposals, our optimization approach reassembles those piecestogether into multiple person instances. It enforces that the composed bodypart regions of each person instance obey constraints on relative sizes, mutualspatial relationships, foreground coverage, and exclusive label assignmentswhen overlapping. Since optimal region assembly is a challenging combinatorialproblem, we present a Lagrangian relaxation method to accelerate the lowerbound estimation, thereby enabling a fast branch and bound solution for theglobal optimum. As output, our method produces a pixel-level map indicatingboth 1) the body part labels (arm, leg, torso, and head), and 2) which partsbelong to which individual person. Our results on three challenging datasetsshow our method is robust to clutter, occlusion, and complex poses. Itoutperforms a variety of competing methods, including existing detector CRFmethods and region CNN approaches. In addition, we demonstrate its impact on aproxemics recognition task, which demands a precise representation of "whosebody part is where" in crowded images.